Method for autonomous detection of crop location based on tool depth and location

ABSTRACT

A method for detecting real lateral locations of target plants includes: recording an image of a ground area at a camera; detecting a target plant in the image; accessing a lateral pixel location of the target plant in the image; for each tool module in a set of tool modules arranged behind the camera and in contact with a plant bed: recording an extension distance of the tool module; and recording a lateral position of the tool module relative to the camera; estimating a depth profile of the plant bed proximal the target plant based on the extension distance and the lateral position of each tool module; estimating a lateral location of the target plant based on the lateral pixel location of the target plant and the depth profile of the plant bed surface proximal the target plant; and driving a tool module to a lateral position aligned with the lateral location of the target plant.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. patentapplication Ser. No. 16/539,390, filed on 13 Aug. 2019, which claims thebenefit of U.S. Provisional Application No. 62/718,330, filed on 13 Aug.2018, both of which are incorporated in their entirety by thisreference.

TECHNICAL FIELD

This invention relates generally to the field of agricultural implementsand more specifically to a new and useful method for autonomouslydetecting crops in an agricultural field while performing agriculturalactivities in the field of agricultural implements.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a flowchart representation of a first method;

FIG. 2 is a schematic representation of an autonomous machine;

FIG. 3 is a flowchart representation of the first method; and

FIG. 4 is a flowchart representation of the first method.

DESCRIPTION OF THE EMBODIMENTS

The following description of embodiments of the invention is notintended to limit the invention to these embodiments but rather toenable a person skilled in the art to make and use this invention.Variations, configurations, implementations, example implementations,and examples described herein are optional and are not exclusive to thevariations, configurations, implementations, example implementations,and examples they describe. The invention described herein can includeany and all permutations of these variations, configurations,implementations, example implementations, and examples.

1. Method

As shown in FIG. 1, a method S100 for detecting real lateral locationsof target plants relative to an autonomous machine 100 can include, atthe autonomous machine 100: autonomously navigating within anagricultural field in Block S110; recording a first image of a groundarea at a ground-facing camera 112 arranged proximal a front of theautonomous machine 100 in Block S120; detecting a first target plant inthe first image in Block S130; accessing a lateral pixel location of thefirst target plant in the first image in Block S132; and, at a depthsensor 116 proximal the front of the autonomous machine 100 and defininga field of view encompassing a subregion of the ground area, estimatinga depth of the subregion of the ground area in Block S140. The methodS100 also includes, for each tool module in a set of tool modulesarranged behind the ground-facing camera and in contact with a plant bedsurface: recording an extension distance of the tool module 130 in BlockS150; and recording a lateral position of the tool module 130 relativeto the ground-facing camera 112 in Block S152. The method S100 furtherincludes: estimating a surface profile of the plant bed surface based onthe extension distance of each tool module 130 in the set of toolmodules 130 and the lateral position of each tool module 130 in the setof tool modules 130 in Block S160; estimating a depth profile based onthe surface profile and the depth of the subregion of the ground area inBlock S110; estimating a real lateral location of the first target plantrelative to the autonomous machine 100 based on the lateral pixellocation of the first target plant and the depth profile of the plantbed surface proximal the first target plant in Block S180; and driving afirst tool module 130 in the set of tool modules 130 to a lateralposition laterally aligned with the real lateral location of the firsttarget plant in Block S190.

One variation of the method S100 can include, at the autonomous machine100: at a first time, recording a first image of a first ground area ata ground-facing camera 112 arranged proximal a front of the autonomousmachine 100 in Block S120; detecting a first target plant in the firstimage in Block S130; and accessing a lateral pixel location of the firsttarget plant in the first image in Block S132. This variation of themethod S100 also includes, at a second time, for each tool module in aset of tool modules arranged behind the ground-facing camera and incontact with a plant bed surface: recording a first extension distanceof the tool module 130 in Block S150; and recording a first lateralposition of the tool module 130 relative to the ground-facing camera 112in Block S152. This variation of the method S100 further includesestimating a first depth profile of the plant bed surface proximal thefirst target plant based on the first extension distance of each toolmodule 130 in the set of tool modules 130 and the first lateral positionof each tool module 130 in the set of tool modules 130 in Block S172;estimating a real lateral location of the first target plant relative tothe autonomous machine 100 based on the lateral pixel location of thefirst target plant and the first depth profile of the plant bed surfaceproximal the first target plant in Block S180; and driving a first toolmodule 130 in the set of tool modules 130 to a lateral positionlaterally aligned with the real lateral location of the first targetplant in Block S190.

Another variation of the method S100 includes, at the autonomous machine100: autonomously navigating within an agricultural field in Block S110;at a first time, recording a first image of a first ground area at aground-facing camera 112 arranged proximal a front of the autonomousmachine 100 in Block S120; detecting a first target plant in the firstimage in Block S130; and accessing a lateral pixel location of the firsttarget plant in the first image in Block S132. This variation of themethod S100 also includes, at a second time, for each tool module in aset of tool modules mounted to a toolbar arranged behind ground-facingcamera and in contact with a plant bed surface: recording an extensiondistance of the tool module 130 relative to the toolbar in Block S150;and recording a lateral position of the tool module 130 relative to thetoolbar in Block S152. This variation of the method S100 furtherincludes: estimating a depth profile of the plant bed surface proximalthe target plant based on the extension distance of each tool module 130in the set of tool modules 130, the lateral position of each tool module130 in the set of tool modules 130, and an inclination of the toolbar inBlock S174; estimating a real lateral location of the first target plantrelative to the ground-facing camera 112 based on the lateral pixellocation of the first target plant and the depth profile in Block S180;and driving a first tool module 130 in the set of tool modules 130 alongthe toolbar to a lateral position laterally aligned with the reallateral location of the first target plant in Block S190.

2. Applications

Generally, the method S100 can be executed by an autonomous farmimplement (hereinafter an “autonomous machine 100”) to automatically:navigate along plant beds including rows of crops in an agriculturalfield; to record images of the plant bed using a ground-facing camera112; to detect target plants in the recorded images; to extract pixellocations of those target plants; to estimate a surface profile of theplant bed surface by utilizing the vertical position (or extension) oftool modules 130 located behind the ground-facing camera 112, whereineach tool module 130 is contacting the plant bed surface (e.g. rollingalong the plant bed); to calculate the real lateral location of a targetplant based on the estimated surface profile; and to laterally actuatethe tool module 130 along the toolbar to intercept the real laterallocation of the target plant in response to longitudinal motion of theautonomous machine 100.

In particular, the autonomous machine 100 can execute Blocks of themethod S100 to accurately calculate the location of a target plant onthe plant bed surface in three-dimensional space relative to theautonomous machine 100 without needing additional depth sensor 116 s(e.g. LIDAR) proximal the ground-facing camera 112 or an additionalcamera for binocular vision to obtain depth information. The autonomousmachine 100 utilizes sensors on the tool modules 130 that performagricultural functions to record depth information regarding the plantbed surface. Thus, the autonomous machine 100 can process images outputby a ground-facing camera 112 in combination with the depth informationaccording to the method S100 in order to achieve high lateral locationalaccuracy for detected target plants passed by the autonomous machine 100during operation. The autonomous machine 100 can then precisely performagricultural functions such as weeding, watering, and fertilizing on thetarget plant via the laterally mobile tool modules 130 installed on theautonomous machine 100 that can align with the calculated laterallocation of the target plant.

In one variation of the method S100, the autonomous machine 100 can alsoinclude a depth sensor 116 (e.g., LIDAR) proximal the front of theautonomous machine 100 and can detect the depth of a subsection of theplant bed surface. The autonomous machine 100 can then estimate theshape of the surface (as a surface profile) according to depthinformation obtained from the set of tool modules 130. Therefore, theautonomous machine 100 can obtain an accurate depth profile across theentire field of view of the ground-facing camera 112 while detecting thedepth of only a small subsection of the ground area within the field ofview of the ground-facing camera 112, thereby reducing the overall costof the autonomous machine due to a reduction in the number of depthsensor 116 s needed to map the depth of the plant bed surface.Furthermore, the autonomous machine 100 can also detect the height ofplants in the agricultural field (e.g., an average height) in order toimprove lateral and longitudinal location estimates for each targetplant.

The autonomous machine 100 can include modular tool modules 130 that maybe exchanged by the user to alternatively perform weeding, watering,seeding, and/or other agricultural functions. The autonomous machine 100accomplishes these agricultural functions by drawing the tool modules130 along a plant bed, which may include two to six rows of plantsacross the width of the autonomous machine 100. Additionally, the toolmodules 130 can be mounted to a laterally-oriented (i.e. perpendicularto the longitudinal axis of the autonomous machine 100 or the forwarddirection of the autonomous machine 100) toolbar. The autonomous machine100 can individually actuate each of the tool modules 130 along thetoolbar, such that the tool modules 130 laterally align with targetplants that are passing under the autonomous machine 100 as theautonomous machine 100 navigates through an agricultural field. Thus,the independently-movable tool modules 130 can intercept target plants(or another identifiable location on the plant bed) and more accuratelyperform weeding, watering, or fertilizing operations.

The autonomous machine 100 also includes a ground-facing camera 112,mounted forward of the toolbar and tool modules 130, that can recordimages of the plant bed. The autonomous machine 100 can then implementcomputer vision techniques to analyze the plant bed and detect targetplants such that the tool modules 130 can effectively weed, water,fertilize, or otherwise operate around the target plant. Once theautonomous machine 100 detects a target plant in the image, theautonomous machine 100 can extract a pixel location of the target plantin the image (e.g. a centroid of the target plant or an approximatelocation of the stem of a target plant). The autonomous machine 100 thencalculates an incident projection in three-dimensional space relative tothe ground-facing camera 112 corresponding to the extracted pixel.

Hereinafter, the term “projection” of a particular pixel represents theazimuthal and radial angles of the center of the field of view of theparticular pixel. For example, after detecting the center of a plant inan image recorded by the ground-facing camera 112, the autonomousmachine 100 can query a mapping or parametric model for the projectionof a particular pixel corresponding to the center of the plant. Themapping or parametric model can, therefore, output radial and azimuthalangles of a ray extending from the camera through the center of theplant (e.g. relative to the camera or another reference point on theautonomous machine 100). Alternatively, the autonomous machine 100 canrepresent the projection mathematically in order to describe the headingand position of the projection in three-dimensional space relative tothe autonomous machine 100, such as in the form of a set of projectivecoordinates, a linear function, or a vector, etc.

To estimate a location of the target plant along the projectioncorresponding to the pixel location of the target plant, the autonomousmachine 100 records extension distances provided by extension sensors(e.g. linear encoders or rotational encoders) that individually measurethe extension distance of the tool modules 130 from the toolbar. Thetool modules 130 are vertically mobile relative to the toolbar via asuspension mechanism so that the tool modules 130 maintain contact withthe surface of the plant bed (e.g. with wheels). The autonomous machine100 records the extension distance for each tool module 130, whichcorresponds with the depth of the plant bed surface at the lateralposition of the tool module 130. Therefore, by simultaneously recordingthe lateral position of each tool module 130 and the extension distanceof each tool module 130, the autonomous machine 100 obtains several datapoints indicating the depth of the surface of the plant bed relative tothe toolbar of the autonomous machine 100. The autonomous machine 100can perform additional processing, such as interpolation and regressionon the extension distances to estimate a surface profile of the plantbed surface at the position of the toolbar.

The autonomous machine 100 can be preprogrammed with calibratedgeometric information relating the exact position and orientation of thetoolbar to the exact position and orientation of the ground-facingcamera 112. In implementations where the toolbar is mobile relative tothe chassis 104 of the autonomous machine 100, calibration can beperformed at multiple orientations of the toolbar. Thus, the autonomousmachine 100 can estimate a depth of the plant bed surface along theprojection of the pixel corresponding to the target plant by waitinguntil the longitudinal location of the target plant is sufficientlyclose to the longitudinal location of the toolbar. When the target plantis close enough, the autonomous machine 100 can estimate the currentsurface profile based on the extension distances of tool modules 130 andthe position of the toolbar to calculate an intersection between theprojection of the target plant and the surface profile and estimate anaccurate lateral location of the target plant.

In one implementation, the autonomous machine 100 can actuate thetoolbar to adjust the height of the toolbar and the tilt of the toolbarto better position the tool modules 130 based on the surface profile ofthe plant bed. For example, each tool module 130 has a maximum and aminimum extension distance defining an extensible range of the toolmodules 130. Thus, the autonomous machine 100 can adjust the tilt of thetoolbar to substantially match an average tilt of the surface profile ofthe plant bed such that the variation between the extension distance ofeach toolbar is minimized. Additionally, the autonomous machine 100 canactuate the toolbar to adjust the height of the toolbar, such that eachtool module 130 is operating at approximately the midpoint of each toolmodule's vertical extensible range. Furthermore, the autonomous machine100 can adjust the inclination of the toolbar in order to compensate forplant bed tilt, while maintaining normal function for the tool modules130.

The autonomous machine 100 is described below as including weedingmodules and executing the method S100 to de-weed an agricultural field.However, the autonomous machine 100 can implement similar methods andtechniques to prepare and then trigger tool modules 130 of othertypes—such as seeding, watering, fertilizing, harvesting, and pesticidemodules—to apply water or fertilizer to target plants and/or applypesticides around these target plants, etc.

3. Autonomous Machine

As shown in FIG. 2, the autonomous machine 100 is configured toautonomously navigate through an agricultural field. The autonomousmachine 100 can thus define a wheeled or tracked vehicle and can includea chassis 104 and a drive unit 106 configured to propel the autonomousmachine 100 forward. The autonomous machine 100 can also include:geospatial position sensors 108 (e.g., GPS) configured to output theautonomous machine 100's location in space; inertial measurement unitsconfigured to output values representing the autonomous machine 100'strajectory; and/or outwardly facing color and/or depth sensor 116 s(e.g., color cameras, LIDAR sensors, and/or structured light cameras,etc.) configured to output images from which the autonomous machine 100can detect nearby obstacles, localize itself within a scene, and/orcontextualize a nearby scene; etc. The autonomous machine 100 can alsoinclude an onboard navigation system configured to collect data from theforegoing sensors, to elect next actions, and to adjust positions ofvarious actuators within the autonomous machine 100 to execute thesenext actions.

3.1 Light Module

The autonomous machine 100 can also include a light module 110 arrangedproximal the front of the autonomous machine 100 in order to preventexternal light from illuminating the plant bed surface thereby improvingclassification and location of target plants. The light module 110 candefine an enclosed volume with a downward-facing opening spanning one ormore crop rows. The light module 110 can also include controllablelighting elements 114 configured to repeatably illuminate a ground areadirectly under the opening of the light module 110.

3.2 Tool Housing

The autonomous machine 100 can further include a tool housing 120arranged behind the light module 110 and configured to house a toolbarand/or one or more tool modules 130 mounted to the toolbar, such asdescribed below.

In one implementation, the tool housing 120 includes a toolbarconfigured to transiently receive one or more tool modules 130 mountedonto the toolbar. The toolbar is positioned laterally relative to theautonomous machine 100, or perpendicular to the direction of forwardmotion (i.e. longitudinal direction) of the autonomous machine 100. Thetoolbar defines a long extrusion of material, such as high strengthsteel or aluminum, configured to physically support one or more toolmodules 130 mounted to the toolbar without significant deformation.Additionally, the toolbar can define a cross section configured to fitwithin an actuating slot on each tool module 130. Therefore, the toolmodules 130 can actuate laterally along the toolbar. The toolbar canspan the entire width of a plant bed (which may include one or moreplant rows) over which the autonomous machine 100 navigates, such thattool modules 130 mounted on the toolbar can laterally align an endeffector of a tool module 130 with successive plants in a row of cropsover which the autonomous machine 100 passes during operation. Forexample, the tool housing 120 of the autonomous machine 100 can includefour (or six) tool modules 130 mounted to the toolbar. To autonomouslyweed a field of crops, each tool module 130 in the autonomous machine100 can be a weeding module. As the autonomous machine 100 passes over afield of crops, these tool modules 130 can be independently-controlledto laterally align with successive target plants in corresponding rowsof crops as these weeding modules selectively upset weeds whilerendering target plants (i.e., crops) substantially undisturbed.

At another time, to water these crops, a user may replace the weedingmodules with watering modules connected to a common water reservoirinstalled on the autonomous machine 100. As the autonomous machine 100navigates along rows of crops, the autonomous machine 100 can:independently control these tool modules 130 to laterally align withtarget plants in its corresponding crop row; and selectively triggereach watering module to dispense water onto target plants in theircorresponding crop rows.

Similarly, to fertilize these crops, a user may replace the tool modules130 with fertilizing modules connected to a common fertilizer reservoirinstalled on the autonomous machine 100. As the autonomous machine 100navigates along rows of crops, the autonomous machine 100 can:independently control the fertilizing modules to laterally align eachfertilizing module with a target plant in its corresponding crop row;and selectively trigger each fertilizing module to dispense fertilizeronto these target plants.

The autonomous machine 100 can also include toolbar actuators couplingthe toolbar to the tool housing 120. In one implementation, the toolbaractuators are coupled to each end of the toolbar, such that theautonomous machine 100 can raise or lower the toolbar. Additionally oralternatively, the toolbar actuators can be actuated independently (e.g.the autonomous machine 100 can actuate a left toolbar actuator upwardwhile actuating a right toolbar actuator downward or by keeping onetoolbar actuator stationary while actuating the second toolbaractuator), thereby imparting a tilt to the toolbar. As such, a toolbaractuator can include a triangulated suspension arm at an end of thetoolbar and a linear actuator also coupled to the end of the toolbar,such that the toolbar can actuate upward and downward while alsochanging the inclination of the toolbar. The autonomous machine 100 caninclude self-actuating tool modules 130 that can adjust theirorientation relative to the toolbar to maintain an upright orientationrelative to the plant bed when the toolbar is inclined.

The autonomous machine 100 can utilize closed-loop controls to actuatethe toolbar. Thus, the toolbar can include linear or rotational encodersconfigured to measure the real position (i.e. height and tilt) of thetoolbar. However, the encoders serve a dual function in that, bymeasuring the real position of the toolbar, they provide the autonomousmachine 100 with information that, in combination with extensiondistance data from encoders on the tool modules 130 can be utilized toestimate a surface profile of the plant bed.

Alternatively, the tool modules 130 are not mounted directly to thetoolbar and instead the tool housing 120 includes a tool receptacle 124:attached to the toolbar; configured to transiently receive one ofvarious tool modules 130; and including a tool positioner 122 configuredto shift the tool receptacle 124 laterally within the tool housing 120in order to laterally align an end effector of a tool module 130—loadedinto the tool receptacle 124—with successive plants in a row of cropsover which the autonomous machine 100 passes during operation. Toautonomously weed a field of crops, each tool receptacle 124 in theautonomous machine 100 can be loaded with a weeding module. As theautonomous machine 100 passes over a field of crops, tool positioners122 in these tool receptacles 124 can be independently-controlled tolaterally align their weeding modules to successive target plants incorresponding rows of crops as these weeding modules selectively upsetweeds while rendering target plants (i.e., crops) substantiallyundisturbed.

At another time, to water these crops, the tool receptacles 124 can beloaded with watering tools connected to a common water reservoirinstalled on the autonomous machine 100. As the autonomous machine 100navigates along rows of crops, the autonomous machine 100 can:independently control tool positioners 122 in these tool receptacles 124to laterally align each watering tool to target plants in itscorresponding crop row; and selectively trigger each watering tool todispense water onto target plants in their corresponding crop rows.

Similarly, to fertilize these crops, the tool receptacles 124 can beloaded with fertilizing tools connected to a common fertilizer reservoirinstalled on the autonomous machine 100. As the autonomous machine 100navigates along rows of crops, the autonomous machine 100 can:independently control tool positioners 122 in these tool receptacles 124to laterally align each fertilizing tool to target plants in itscorresponding crop row; and selectively trigger each fertilizing tool todispense fertilizing onto these target plants.

Tool receptacles 124 in the tool housing 120 can be similarly loadedwith: fertilizing tools; pesticide/herbicide tools; thinning or cullingtools; seeding tools; and/or harvesting tools; etc.

3.3 Cameras

The autonomous machine 100 can also include various cameras or otheroptical sensors arranged inside the light module 110 and inside the toolhousing 120 and configured to record images of the plant bed passingunder the light module 110 and the tool housing 120 as the autonomousmachine 100 autonomously navigates along crop rows within anagricultural field.

In one implementation, the autonomous machine 100 includes one or moreground-facing camera 112S (e.g., a high-resolution, high-speed RGB orCMYK camera or multi-spectral imager, and/or LIDAR) arranged in thelight module 110, defining a field of view spanning all or a portion ofthe opening of the light module 110, and configured to record images ofareas of the plant bed entering the light module no from the front ofthe autonomous machine 100 (i.e., areas of the plant bed that theautonomous machine 100 is navigating over). The autonomous machine 100can then analyze these images to detect and distinguish crops (or“target plants”) from weeds, to calculate locations of target plantswith a relatively high degree of accuracy and repeatability inthree-dimensional space and relative to the orientation and position ofthe ground-facing camera 112, and/or to extract qualities of thesetarget plants (e.g., pest presence, fertilizer burns, nutrient or waterdeficiency, etc.).

3.4 Depth Sensor 116

In one variation, the autonomous machine 100 can include a ground-facingdepth sensor 116 such as a LIDAR, time-of-flight camera, or any otherdepth sensing device. The autonomous machine 100 can include the depthsensor 116 mounted within the light box of the autonomous machine 100proximal the ground-facing camera 112. Therefore, the depth sensor 116can define a field of view within the field of view of the ground-facingcamera 112. However, in one implementation, the depth sensor 116 candefine a field of view covering a subsection of the ground areaencompassed by the field of view of the ground-facing camera 112 despitebeing positioned proximal to the ground-facing camera 112 (e.g., due toa narrower field of view of the depth sensor 116).

However, the depth sensor 116 can be positioned at any location withinthe light box of the autonomous machine 100.

4. Tool Modules

As described above, the autonomous machine 100 can include one or moretool modules 130 mounted to the toolbar and configured to perform aparticular agricultural function, such as weeding, watering, orfertilizing. In one implementation, each type of tool module 130 sharesa similar structure including: an actuating slot assembly configured toclamp onto the toolbar and to shift laterally relative the toolbar, apivot coupled to the actuating slot assembly, an end effector to performthe agricultural function of the tool module 130 (e.g. a weeding bladesystem, water or fertilizer dispensing system, etc.), and a suspensionsystem 134, which suspends the end effector at a consistent height aboveor below the adjacent plant bed surface.

In one implementation, the actuating slot assembly of the tool module130 can include wheels (which can be grooved or ratcheted to preventslippage along the toolbar) configured to rotate as the tool module 130proceeds laterally along the toolbar. The tool module 130 can include asecond set of wheels below the toolbar to prevent dislodgement of thetool module 130 from the toolbar by effectively clamping onto thetoolbar from above and below the toolbar. Additionally or alternatively,the wheels of the tool module 130 can fit into corresponding grooves onthe toolbar to prevent the wheels from sliding off the toolbar. Theactuating slot assembly of the tool module 130 can also include anencoder, such as an optical linear encoder, which can directly determinethe lateral location of the tool module 130 on the toolbar.Additionally, the actuating slot assembly can be configured to open andclose the actuating slot, such that the tool module 130 can be mountedand removed from the toolbar.

Alternatively, the tool modules 130 can be rigidly mounted to thetoolbar via a clamp or other mechanism and can include an integratedactuator configured to shift the tool module 130 laterally relative tothe clamp.

In one implementation, the actuating slot assembly or other actuatingmechanism is coupled to the end effector and passive suspension system134 via a pivot point. Thus, the tool module 130 can maintain itsorientation despite changes in tilt of the toolbar. For example, if thetoolbar is inclined at two degrees, the pivot point of each tool module130 mounted to the toolbar can counteract that tilt by rotating twodegrees in the opposite direction, thereby maintaining a verticalorientation. In one implementation, the pivot point is passive androtation about the pivot point is caused by the force of gravity on theend effector and suspension system 134. Alternatively, the end effectorand suspension system 134 can rotate about the pivot point viahydraulic, pneumatic, or electromechanical actuation.

The tool module 130 includes a suspension system 134, which suspends theend effector of the tool module 130 at a specific height above or belowthe surface of the plant bed. The suspension system 134 can include awheel or sliding apparatus (e.g. a ski) configured to contact and followthe adjacent plant bed surface, such that the end effector—coupled tothe suspension system 134—remains at a constant depth relative the plantbed surface as the autonomous machine 100 navigates through theagricultural field. In one example, the suspension system 134 includestwo wheels arranged on either side of the end effector and sufficientlyoffset from the end effector, such that the autonomous machine 100 canalign the end effector with target plants without running over thetarget plant with the one or more wheels of the tool module 130. In oneimplementation, the suspension system 134 is passive and includes aspring and damper system which extends under the sprung weight of thetool module 130 not supported by the toolbar to which the tool module130 is mounted (e.g. the end effector, suspension components, andwheels). The sprung weight of the tool module 130 causes the suspensionsystem 134 to extend toward the plant bed until the wheel of the toolmodule 130 contacts the plant bed and prevents further extension of thesuspension system 134. Alternatively, the suspension system 134 can behydraulically, pneumatically, or electromechanically extendable. Forexample, the suspension system 134 can include a hydraulic actuatorwhich extends the end effector and wheels of the tool module 130 towardthe plant bed until the wheels come into contact with the plant bedsurface. In one implementation, the suspension system 134 includes apneumatic piston and spring system wherein the autonomous machine 100can adjust the force with which the end effectors of the tool modules130 are pressed into the ground, thereby enabling precise adjustment ofend effector depth and/or pressured exerted upon the plant bed.

Furthermore, the suspension system 134 can include an encoder todetermine the extension distance of the suspension system 134 relativeto the toolbar. For example, the encoder can indicate that thesuspension system 134 is extended ten centimeters past its homeposition. Thus, the autonomous machine 100 can record tool position dataincluding: the height and tilt of the toolbar via encoders in thetoolbar actuators; and, for each tool module 130, the lateral positionvia the actuating slot encoders and the extension distance of thesuspension system 134. The autonomous machine 100 can then use the toolposition data in combination with images from the ground-facing camera112 to accurately locate target plants on the plant bed surface, asdiscussed below.

However, the tool modules 130 can record extension distances from thetoolbar to the plant bed using any other device, for example cameras,infrared or other proximity sensors, etc. mounted to each tool module130.

4.1 Weeding Module

In one variation, the autonomous machine 100 includes weeding modulesmounted on the toolbar. In one implementation, the weeding module caninclude a pair of blades 132 and a blade actuator configured totransition the blades 132 between open and closed positions. In thisimplementation, the blades 132: can define curved, cantilevered sectionsextending from driveshafts suspended from the toolbar; and submerged intopsoil, such as configured to run 0-60 millimeters below the plant bedsurface while the autonomous machine 100 traverses an agricultural fieldin order to dislodge weeds from topsoil. The blades 132 can also begeared or otherwise driven together by the blade actuator—such as anelectromagnetic rotary motor or a pneumatic linear actuator—such thatthe blades 132 open and close together.

In the closed position by default, tips of blades 132 can come intocontact or nearly into contact such that the blades 132 form acontinuous barricade across the width of the weeding module; the blades132 in the closed position can thus displace topsoil and tear weeds outof the topsoil across the full lateral span of the blades 132 in theclosed position. In this implementation, the pair of blades 132 can alsobe vertically offset relative to one another, thereby enabling the tipsof the blades 132 to overlap to ensure a continuous barricade across thewidth of the weeding module in the closed position.

However, when opened by the blade actuator, tips of the blades 132spread apart, thereby forming an open region between the tips of theblades 132. The blade actuator can therefore transition the blades 132to the open position in order to form a gap between the blades 132:sufficient to fully clear the stalk of a target plant passing under theweeding module; sufficient to minimally disrupt topsoil around thetarget plant; but sufficiently closed to dislodge other non-targetplants (e.g., weeds) immediately adjacent the target plant from thetopsoil as the autonomous machine 100 autonomously navigates past thetarget plant.

In this implementation, in the open position, the blade actuator canopen the blades 132 by a distance matched to a type and/or growth stageof crops in the field. For example, the target open distance between theblades 132 in the open position can be set manually by an operator priorto dispatching the autonomous machine 100 to weed the agriculturalfield. In this example, the operator can select the target open distancebetween the tips of the primary blade in the open position from adropdown menu, such as: 20 mm for lettuce at two weeks from seeding; 30mm for lettuce at three weeks from seeding; 40 mm for lettuce at fourweeks from seeding; and 50 mm spread for lettuce after five weeks fromseeding and until harvest. Alternatively, to avoid disrupting a smalltarget plant with shallow roots but to improve weeding accuracy for moremature plants with deeper root structures, the operator can select thetarget open distance between the tips of the primary blade in the openposition of: 50 mm for lettuce at two weeks from seeding; 40 mm forlettuce at three weeks from seeding; 30 mm for lettuce at four weeksfrom seeding; and 20 mm spread for lettuce after five weeks from seedingand until harvest. Alternatively, the autonomous machine 100 oraffiliated support infrastructure can automatically select thesedistances based on the size of each plant, as estimated by theground-facing camera 112. The blade actuator can then implement thesesetting during a next weeding operation at the field, as describedbelow.

In particular, the blade actuator can be configured to retain the blades132 in the closed position by default such that the blades 132 displacetopsoil and tear weeds out of the topsoil across the full lateral spanof the blades 132 as the autonomous machine 100 navigates along a croprow. However, in this example, upon nearing a target plant theautonomous machine 100 can trigger the blade actuator to open the blades132 by the target open distance to permit the target plant to passthrough the weeding module substantially undisturbed; once the targetplant passes through the opening in the blades 132, the autonomousmachine 100 can trigger the blade actuator to return to the closedposition.

5. Operation

When dispatched to an agricultural field to perform a weeding operation,the autonomous machine 100 can autonomously navigate along crop rows inthe agricultural field to detect and track target plants and toselectively actuate the weeding modules to dislodge plants other thantarget plants from the topsoil.

In particular, the method S100 is described below as executed by theautonomous machine 100 when loaded with a weeding module. For theautonomous machine 100 that includes multiple weeding modules, theautonomous machine 100 can execute multiple instances of Blocks of themethod S100 simultaneously in order: to detect target plants in multiplediscrete crop rows; to independently reposition these weeding modulesinto lateral alignment with target plants in their corresponding croprows; and to selectively trigger their blade actuators to open and closein order to upset weeds while leaving target plants in these rowssubstantially undisturbed. Additionally, the autonomous machine 100 canperform the method S100 while performing other agricultural functionsand while loaded with other tool modules 130. However, the method S100is described with reference to weeding for ease of description.

5.1 Navigation

In one implementation, the autonomous machine 100: includes a set ofgeospatial position sensors 108 (e.g., a GPS sensors); and tracks itsabsolute position and orientation within a geospatial coordinate systembased on outputs of these geospatial position sensors 108. Inpreparation for a weeding operation within an agricultural field, theperimeter or vertices of the agricultural field can be defined withinthe geospatial coordinate system and then loaded onto the autonomousmachine 100. The longitudinal direction and lateral offset of crop rowsin this agricultural field, start and stop locations (e.g., within thegeospatial coordinate system), a target ground speed, and other relevantdata can be similarly loaded onto the autonomous machine 100.

Once the autonomous machine 100 is dispatched to this agricultural fieldand once a weeding cycle by the autonomous machine 100 is subsequentlyinitiated by an operator (e.g., locally or remotely), the autonomousmachine 100 can, in Block S110: navigate to the specified start location(e.g., around rather than through the georeferenced boundary of theagricultural field); orient itself into alignment with the longitudinaldirection of a first set of crop rows at the start location; andaccelerate to the target ground speed parallel to the first set of croprows. While traversing the first set of crops rows in a plant bed, theautonomous machine 100 can: record and process images recorded by theground-facing camera 112 to detect plants entering the light module nofrom the front of the autonomous machine 100; distinguish target plantsfrom weeds and other ground features, as described below; andinterpolate crop rows between sequential target plants in this first setof crop rows. The autonomous machine 100 can then implement closed-loopcontrols to steer left or steer right in order to maintain the first setof crop rows underneath the autonomous machine 100 and within range ofthe tool modules 130 of the autonomous machine 100. The autonomousmachine 100 can additionally or alternatively detect crop rows throughimages recorded by outwardly-facing cameras on the front of theautonomous machine 100 and align itself to these crop rows accordingly.

Upon reaching the georeferenced boundary of the agricultural field, theautonomous machine 100 can autonomously execute a reverse-offsetmaneuver to turn 180° and align itself with a second set of croprows—offset from the first set of crop rows by an effective width of thetool housing 120 (e.g., by four crop rows for the tool housing 120loaded with four weeding modules). For example, the autonomous machine100 can execute a U-turn maneuver responsive to both GPS triggers andoptical features indicative of the end of the crop row in imagesrecorded by various cameras in the autonomous machine 100. Theautonomous machine 100 can again: accelerate to the target ground speedparallel to the second set of crop rows; maintain the second set of croprows centered within the width of the autonomous machine 100; and repeatthe reverse-offset maneuver to align itself with a third set of croprows upon reaching the opposing georeferenced boundary of theagricultural field. The autonomous machine 100 can repeat theseprocesses until the autonomous machine 100 has traversed the entirety ofthe specified area of the agricultural field, then autonomously navigateback to a stop location, and finally enter a standby mode.

However, the autonomous machine 100 can implement any other method ortechnique to track its location and orientation, to autonomouslynavigate across an agricultural field, and to maintain itself inalignment with rows of crops during a weeding operation.

5.2 Default Weeding Module Operation

As described above, the autonomous machine 100 can maintain the blades132 in the closed position by default in order to upset all non-targetplants in the path of the blades 132 as the autonomous machine 100navigates along a row of crops.

5.3 Plant Detection and Identification

While navigating along the row of crops in Block S110, as shown in FIG.1, the autonomous machine 100 can regularly record images through one ormore ground-facing camera 112S (e.g., RGB color or multispectralcameras) arranged in the light module 110, such as at a rate of 24 Hz,in Block S120. Upon receipt of a first entry image recorded by theground-facing camera 112(s) at a first time, the autonomous machine 100can implement computer vision techniques to: detect and extract featuresin the first entry image; and to identify these features as representingtarget plants, weeds, soil, or other non-target matter in Block S130.For example, the autonomous machine 100 can implement template matching,object recognition, or other plant classifier or computer visiontechniques to detect plant matter in the first entry image and todistinguish a first target plant from weeds in the first entry image(e.g., based on plant color(s), leaf shape, and/or size, etc.). Theautonomous machine 100 can additionally or alternatively implement deeplearning techniques (e.g., convolutional neural networks) to collectthese data.

Once a target plant is identified in an image from the ground-facingcamera 112, the autonomous machine 100 can represent the identifiedtarget plant as a collection of pixels in the image. The autonomousmachine 100 can store the collection of pixels in an array or any otherdata structure.

However, the autonomous machine 100 can implement any other method ortechnique to detect and distinguish a target plant from other featuresof the plant bed represented in the first entry image and to determine aposition of the stalk of the plant in the first entry image.

5.4 Depth Imaging

In one variation, as shown in FIG. 3 the autonomous machine 100 can alsodetect a depth of subregion of the ground area of the plant bed withinthe light module 110 in Block S140, while concurrently recording imagesof target plants in Block S120. More specifically, the autonomousmachine 100 can: at the depth sensor 116, record a depth image of thesubregion of the ground area; identify a ground region within the depthimage; and average the depth of the ground region to calculate the depthof the subregion of the ground area. By recording a depth image of asubregion of the ground area, the autonomous machine 100 can provide anaccurate point of reference for the depth of the plant bed below theautonomous machine 100 and can adjust the surface profile (obtained viaextension distances and lateral positions of tool modules 130 of theautonomous machine 100), which spans a longer lateral distance such thatthe absolute depth of a corresponding region of the surface profilematches the depth of the plant bed measured by the depth sensor 116.Thus, by incorporating depth information from both the depth sensor 116and the tool modules 130, the autonomous machine 100 can obtain anaccurate and wide depth profile for the plant bed and achieve accuratelateral and/or longitudinal position estimates for target plants.

In order to accurately identify the depth of the plant bed surface froma depth image, the autonomous machine 100 can identify regions of thedepth image occupied by plants or other objects and exclude theseregions from the calculation of the average depth of the plant bed (i.e.identifying a ground region of the depth image). In one implementation,the autonomous machine 100 can identify regions of the depth imageoccupied by plants or other obstacles via statistical methods such asoutlier detection, edge detection, classification by a neural network orother machine learning technique, etc. Alternatively, the autonomousmachine 100 can perform sensor fusion by overlaying image data onto thedepth image and identifying regions of the depth image that correspondto particular plants or other objects identified in images of the groundarea.

The autonomous machine 100 can then remove the identified regions fromthe depth image and detect a depth of the ground region within the depthimage. In one implementation, the autonomous machine 100 calculates asingle depth value by averaging the depth of the ground region of thedepth image corresponding to the plant bed. The autonomous machine 100defines this point depth value laterally within the same coordinatesystem defined by the ground-facing camera 112. Alternatively, theautonomous machine 100 calculates a surface that best fits the groundregion of the depth image. Therefore the autonomous machine 100 canadjust a surface profile calculated according to extension distances ofthe tool modules 130 of the autonomous machine 100 to match the surfaceor point depth detected by the depth sensor 116.

However, the depth sensor 116 can detect the depth of the plant bed inany other way.

5.4.1 Plant Height Detection

In one implementation, as shown in FIG. 3, the autonomous machine 100can also detect the height of target plants based on depth imagesrecorded via the depth sensor 116 of the autonomous machine 100. Morespecifically, the autonomous machine 100 can: identify a plant-occupyingregion of within the depth image; and calculate a plant height based onthe plant-occupying region of the depth image.

In this implementation, the autonomous machine 100 can identify regionsof a depth image corresponding to a target plant as described above(e.g., via statistical methods or via sensor fusion with images recordedby the ground-facing camera 112). The autonomous machine 100 can thenanalyze these regions to determine a smoothed peak height of the region,which may roughly correspond to the height of a single target plant(i.e. a plant height). The autonomous machine 100 can then record thevalue of this height across a number of target plants and calculate arunning average or other measure of central tendency of the height oftarget plants within the agricultural field.

By obtaining an approximate height of target plants within theagricultural field, the autonomous machine 100 can more accuratelyestimate the lateral position of the base of the target plant andtherefore better position tool modules 130 to perform agriculturaloperations on or around the target plant.

5.5 Target Plant Location Estimation

Once the autonomous machine 100 has identified a target plant in animage, the autonomous machine 100 can: identify a pixel locationcorresponding to a stalk location of a target plant; identify aprojection in two or three-dimensional space corresponding to the pixellocation; estimate a surface profile based on extension distances oftool modules 130 of the autonomous machine 100; estimate a depth profilebased on the surface profile; and calculate a location of the targetplant based on the depth profile, the projection corresponding to thepixel location of the target plant, and/or the plant height of targetplants.

5.5.1 Pixel Location Identification

Once the autonomous machine 100 identifies a target plant in the firstentry image, the autonomous machine 100 can extract the pixel locationof the first target plant in first entry image in Block S132; and mapthe extracted pixel location to a projection of the target plant.

The autonomous machine 100 identifies a pixel location approximating alocation of a stalk of the target plant within the field of view of thecamera at the time the image was recorded. For example, the autonomousmachine 100 can calculate a centroid of the set of pixels identified asthe target plant. Alternatively, the autonomous machine 100 can performadditional computer vision techniques to approximate the pixel locationof the stem from the set of pixels identified as the target plant. Theautonomous machine 100 can then extract the coordinates of theidentified pixel.

5.5.2 Projection Identification

The autonomous machine 100 can then calculate a projection,corresponding to the extracted pixel coordinates. The calculatedprojection therefore corresponds to an estimated heading, inthree-dimensional space, of the stalk of the target plant relative tothe ground-facing camera 112. The autonomous machine 100 can store amapping of each pixel location in the field of view of the camera to aprojection corresponding to that pixel. Alternatively, the autonomousmachine 100 can include a parametric model, which takes in pixelcoordinates and outputs the projection corresponding to the pixelcoordinates.

The autonomous machine 100 can store the projection corresponding to thepixel representing a target plant using any suitable mathematicaldescription (e.g., projective coordinates, vector descriptions, linearfunctions, etc.). The autonomous machine 100 can also store an origin ofa projection in three-dimensional space. Alternatively, the autonomousmachine 100 can store only a lateral component of the projection withwhich to estimate a lateral position of a target plant.

5.5.3 Surface Profile Estimation

In order to estimate the location of the target plant along theprojection corresponding to the pixel representing the target plant, theautonomous machine 100 can estimate the depth of the plant bed at thelocation of the target plant. In order to estimate the depth of theplant bed proximal the target plant, the autonomous machine 100 canfirst estimate a surface profile of the plant bed which defines therelative shape of the plant bed surface (as opposed to its absolutedepth magnitude). Once the autonomous machine 100 has estimated thesurface profile of the plant bed, the autonomous machine 100 can locatethe surface profile in three-dimensional space (i.e. at a depth belowthe autonomous machine 100 and relative to the ground-facing camera112), by aligning the surface profile with the depth detected by thedepth sensor 116 of the autonomous machine 100. Alternatively, inimplementations of the autonomous machine 100 that do not include adepth sensor 116, the autonomous machine 100 can use the surface profileas the depth profile without referencing any additional depthmeasurements.

The autonomous machine 100 estimates a surface profile of the plant bedsurface by: recording an extension distance in Block S150 and a lateralposition for each tool module 130 in Block S152 and recording a heightand/or a tilt of the toolbar. Because each tool module 130 extends tocontact the surface of the plant bed and is mounted to the toolbar, theautonomous machine 100 can virtually plot the depth of the plant bedsurface at the lateral position of each of the tool modules 130 giventhe relative positions of the ground-facing camera 112, the toolbar, andeach tool module 130. For example, if the autonomous machine 100includes six tool modules 130, the autonomous machine 100 can plot sixpoints corresponding to the depth of the plant bed surface at a givenlateral position. In one implementation, the autonomous machine 100performs a linear regression of the plotted depth points to estimate asurface profile. Alternatively, the autonomous machine 100 can utilizehigher-order polynomial regression, or a linear, polynomial, or splineinterpolation between each point, to estimate a surface profile in BlockS160.

Thus, the autonomous machine 100 can: for each tool module 130 in theset of tool modules 130, define a contact point of the tool module 130at an intersection of the first extension distance of the tool module130 and the first lateral position of the tool module 130 to generate aset of contact points (i.e. in a longitudinal plane relative to theautonomous machine 100); and interpolate between the set of contactpoints to generate the surface profile of the plant bed (or depthprofile in implementations of the autonomous machine 100 not including adepth sensor 116).

Because each tool module 130 is aligned longitudinally (i.e. in the samelongitudinal plane), these surface points constitute a two-dimensionalprofile of the plant bed surface at the longitudinal position of thetool modules 130. However, because plant bed surfaces are relativelyconsistent in the longitudinal direction, the autonomous machine 100 canpropagate forward the two-dimensional surface profile at the location ofthe tool modules 130 to estimate the surface profile of the plant bedsurface along the projections of pixels representing the target plant.The autonomous machine 100 can virtually plot an extendedthree-dimensional surface against the projection corresponding to thepixel representing the target plant. The autonomous machine 100 can thenestimate a location of the target plant based on the intersection inthree-dimensional space of the estimated plant bed surface and theprojection corresponding to the target plant in Block S180. Once anintersection between the estimated plant bed surface and the projectionof the target plant is calculated, the autonomous machine 100 canextract a lateral coordinate and/or a longitudinal coordinate of theintersection point.

In one implementation, the autonomous machine 100 can calculate anaverage surface profile (or depth profile, depending on theimplementation) based on the extension distances and lateral locationsof the tool modules 130 over successive samples as the autonomousmachine 100 navigates along a crop row in an agricultural field. Thisautonomous machine 100 can associate this average depth profile with aparticular crop row and reset the depth profile upon navigating to a newcrop row. Therefore, the autonomous machine 100 can: record a secondimage of a second ground area at the ground-facing camera 112, thesecond ground area located within the first crop row; detect a secondtarget plant in the second image; access a lateral pixel location of thesecond target plant in the second image; and for each tool module 130 inthe set of tool modules 130, record a second extension distance of thetool module 130 and a second lateral position of the tool module 130relative to the ground-facing camera 112. The autonomous machine 100 canthen: estimate a second depth profile of the plant bed surface proximalthe target plant based on the second extension distance of each toolmodule 130 in the set of tool modules 130 and the second lateralposition of each tool module 130 in the set of tool modules 130;estimate an average depth profile based on the first depth profile andthe second depth profile, the average depth profile associated with thefirst crop row; estimate a real lateral location of the second targetplant relative to the autonomous machine 100 based on the lateral pixellocation of the second target plant and the average depth profile; drivethe first tool module 130 in the set of tool modules 130 to a lateralposition laterally aligned with the real lateral location of the secondtarget plant; and, in response to lateral alignment of the lateralposition of the first tool module 130 and the real lateral location ofthe second target plant and in response to longitudinal alignment of alongitudinal position of the tool module 130 and a real longitudinallocation of the second target plant, trigger the first tool module toexecute an agricultural operation on the second target plant.

5.5.4 Depth Profile Estimation

As shown in FIG. 4, the autonomous machine 100 can calculate a depthprofile based on the surface profile and the depth of the subregion ofthe ground area in Block S170. More specifically, the autonomous machine100 can: identify a coincident section of the surface profile laterallycoincident with the subregion of the ground area; and define the depthprofile as a shifted version of the surface profile comprising thecoincident section with an average depth equal to the depth of thesubregion of the ground area.

In particular, upon estimating a surface profile based on a set ofextension distances and a set of lateral locations corresponding to toolmodules 130, the autonomous machine 100 can virtually shift this surfaceprofile to a more accurate depth relative to the autonomous machine 100by virtually aligning the surface profile with a depth reading or depthdetected via the depth sensor 116. Thus, in some implementations, theautonomous machine 100 can assign a lateral position to a depth of theplant bed detected by the depth sensor 116. In one implementation, thelateral position of the depth of the plant bed is calculated as aweighted average of the pixels of the depth image included in thecalculation for the depth measurement. Alternatively, the autonomousmachine 100 defines location as the lateral location of the depth sensor116. In yet another alternative, the autonomous machine 100 cancalculate a best fit surface in three-dimensional space based on theground region of the depth image. This best fit surface thereforeincludes lateral, longitudinal, and vertical information to locate it inthree-dimensional space relative to the ground-facing camera 112.

Therefore, the autonomous machine 100 can virtually shift the surfaceprofile to align with the depth information provided by the depth sensor116. In implementations wherein the autonomous machine 100 calculates asingle depth value representing the depth of the plant bed, theautonomous machine 100 matches a lateral location of the surface profilewith a lateral location of the depth value and sets the depth of thesurface profile to equal the measured depth value at the laterallocation. Alternatively, the autonomous machine 100 can shift thesurface profile to minimize differences between the surface profile anda best fit surface calculated by the autonomous machine 100.

5.5.5 Location Calculation

In Block S180, the autonomous machine 100 calculates the lateralposition of the plant by extracting the lateral coordinate of anintersection between the depth profile of the plant bed surface and alateral component of the projection corresponding to the pixelrepresenting the target plant, thereby estimating the location of thetarget plant in only two dimensions (vertical and lateral). Thistwo-dimensional approach reduces processing time when compared to theaforementioned three-dimensional approach. In this implementation, theautonomous machine 100 can estimate the longitudinal position of thetarget plant based on the image without the depth profile.Alternatively, the autonomous machine 100 can project forward thetwo-dimensional depth profile to generate a three-dimensional depthprofile and calculate an intersection of the projection and the depthprofile in three-dimensional space, thereby estimating both a reallateral and a real longitudinal location of the target plant.

In yet another alternative implementation, the autonomous machine 100can: access the target plant height detected via the depth sensor 116and subtract the target plant height from the depth profile to create amodified depth profile. Thus, the modified depth profile represents thelocations of tops of target plants, which are typically the part of atarget plant visible in images recorded by the autonomous machine 100(and therefore an accurate point of the target plant from which todetermine a location of the target plant).

Thus, the autonomous machine 100 can: access a ray (i.e. a projection)in three-dimensional space relative to the ground-facing camera 112corresponding to the lateral pixel location of the first target plant;subtract the plant height from the depth profile to calculate anapproximate depth of the first target plant; and estimate the reallateral location along the ray, the real lateral location correspondingto the approximate depth of the first target plant.

In one implementation, the autonomous machine 100 can record successiveimages of the plant bed while the target plant passes underneath thelight module 110, thereby continuously updating the projectioncorresponding to the pixel representing the target plant. Additionally,in this implementation, the autonomous machine 100 can update the depthprofile by recording concurrent extension distances from the toolmodules 130. As the target plant approaches the tool modules 130, theestimation of the depth profile of the plant bed surface at the targetplant may improve because the target plant is closer to the tool modules130. In one implementation, the autonomous machine 100 can continue toupdate the depth profile and re-estimate the location of the targetplant even when the plant has left the field of view of theground-facing camera 112 and is located between the light module no andthe tool modules 130. Therefore, the autonomous machine 100 can continueupdating the location of the target plant with additional accuracy untilthe tool modules 130 reach the target plant. Concurrent with each updateto the target plant's location, the autonomous machine 100 can drive thecorresponding tool module 130 to laterally align with each updatedlocation. Thus, as the location estimate of the target plant improves,the lateral offset between the end effector and the location of thetarget plant typically decreases.

In one implementation, the autonomous machine 100 can simultaneouslytrack the location of multiple target plants in several crop rows (e.g.four or six), utilizing the method described above to estimate thelocation of each target plant. In one implementation, the autonomousmachine 100 stores the estimated location of each target plant in amatrix, while also storing a representation of a projectioncorresponding to the pixel representing the target plant in associationwith the estimated location. The autonomous machine 100 can representthe location of a target plant in any suitable coordinate system (e.g.cartesian, spherical, cylindrical). The autonomous machine 100 canutilize image tracking techniques to track each target plant betweensuccessive images and update the location estimate corresponding to eachtarget plant accordingly.

Therefore, the autonomous machine 100 can update the location of eachtarget plant represented in the location matrix based on the mostrecently recorded plant bed surface data from the tool modules 130 andthe stored projection corresponding to each pixel representing eachtarget plant. Thus, even when a target plant is out of view of theground-facing camera 112, the autonomous machine 100 can use the lastrecorded projection corresponding to the last pixel representing thetarget plant to estimate the real location of the target plant.Therefore, the autonomous machine 100 can record an image at a firsttime, and calculate a location of the target plant at a second time,wherein the second time is delayed from the first time based on alongitudinal location of the ground-facing camera 112 relative to alongitudinal location of the first tool module 130 and a speed of theautonomous machine 100. More specifically, the autonomous machine 100can: estimate an intermediate depth profile of the plant bed surface atthe longitudinal location of the set of tool modules 130; and projectthe intermediate depth profile to the longitudinal location of the firsttarget plant to estimate the first depth profile.

Thus, the autonomous machine 100 can continue to recalculate the laterallocation of the target plant until the tool modules 130 longitudinallyalign with the target plant. Alternatively, the autonomous machine 100can concurrently calculate a location of the target plant immediatelysucceeding identifying the target plant and forgo additional estimationbased on updated depth profiles as the autonomous machine traverses theplant bed. Thus, in one implementation, the aforementioned second timecan immediately succeed the first time. More specifically, theautonomous machine can: estimate an intermediate depth profile of theplant bed surface at the longitudinal location of the set of toolmodules; and project the intermediate depth profile forward proximal thetarget plant to estimate the first depth profile.

In one implementation, the autonomous machine 100 can extract a lateralpixel location of the target plant such that the projection isrepresented by an angle relative to the ground-facing camera 112, asopposed to a two-dimensional representation of a projection inthree-dimensional space. In one implementation, the autonomous machine100 extracts the lateral pixel location when the target plant issubstantially longitudinally aligned with the ground-facing camera 112,based on the longitudinal pixel location of the target plant.

However, the autonomous machine 100 can implement any other method forestimating a lateral and/or longitudinal location of the target plantutilizing depth measurement from tool modules 130.

6. Target Plant Tracking

Once the location coordinates (or lateral location coordinate) of thetarget plant have been estimated, the autonomous machine 100 canlaterally align an end effector of a tool module 130 to perform anagricultural function, such as weeding at or immediately around thetarget plant, as described below. The autonomous machine 100 tracks thelocation of the target plant in three-dimensional space based on thespeed and direction of the autonomous machine 100, such that the endeffector can be laterally aligned at the time the tool module 130reaches the target plant.

The autonomous machine 100 tracks the location of the target plant intwo-dimensional space (or three-dimensional space) relative to theautonomous machine 100 based on changes in the global position andorientation of the autonomous machine 100, such that the autonomousmachine 100 can: laterally align an end effector of a tool module 130with the target plant at the time the tool module 130 reaches the targetplant; track the longitudinal location of the target plant; and detectlongitudinal alignment between the blades 132 of the weeding module 130and the opening location corresponding to the target plant.

The autonomous machine 100 can recalculate the location of each targetplant or other relevant location relative to the autonomous machine 100periodically (e.g., 30 times a second) based on the last or mostaccurately calculated location for the target plant (e.g., from amongsta number of images). Each instance in which the autonomous machine 100recalculates the relative locations is hereinafter referred to as a“frame.” After calculating the various tracked locations related totarget plants, the autonomous machine tracks its change in globalposition and/or its change in global orientation since the most recentframe. The autonomous machine 100 can then apply spatial transformations(e.g., rotations and/or translations) to the coordinates defining thetracked locations and generate a new set of locations based on theresult of the spatial transformations. In this manner, the autonomousmachine 100 repeatedly updates the tracked locations relative to theautonomous machine 100.

In one implementation, the autonomous machine 100 can simultaneouslytrack the location of multiple target plants in several crop rows (e.g.four or six), utilizing the method described above to estimate thelocation of each target plant. In one implementation, the autonomousmachine 100 stores the estimated location of each target plant in amatrix. The autonomous machine 100 can represent the location of atarget plant in any suitable coordinate system (e.g. cartesian,spherical, cylindrical). The autonomous machine 100 can utilize imagetracking techniques to track each target plant between successive imagesand update the location estimate corresponding to each target plantaccordingly.

In one implementation, the autonomous machine 100 can record successiveimages of the plant bed while autonomously navigating such that thetarget plant passes underneath the light module 110, therebycontinuously updating the pixel projection corresponding to the pixelrepresenting the target plant. Therefore, the autonomous machine 100 cancontinue updating the location of the target plant until the toolmodules 130 reach the target plant. Concurrent with each update to thetarget plant's location, the autonomous machine 100 can actuate thecorresponding tool module 130 to laterally align with each updatedlocation as further described below.

Therefore, the autonomous machine 100 can update the location of eachtarget plant represented in the location matrix based on the mostrecently recorded image of each target plant. Thus, even when a targetplant is out of view of the front camera 112, the autonomous machine 100can use the last location of the target plant to track the target plant.

By updating the location of each target plant represented in thelocation matrix or via any other two- or three-dimensional trackingmethod, the autonomous machine 100 can: access a longitudinal pixellocation of the first target plant in the first image; calculate a reallongitudinal location of the first target plant relative to theground-facing camera 112 based on the longitudinal pixel location of thefirst target plant and the depth profile; track the real longitudinallocation of the first target plant relative to a longitudinal locationof the first tool module 130 while autonomously navigating within theagricultural field; and, in response to detecting alignment between thereal longitudinal location of the first target plant and thelongitudinal location of the first tool module 130, execute a weedingoperation proximal the first target plant.

7. Weeding Module Positioning

After the autonomous machine 100 calculates a location of a target plantbased on an entry image, the autonomous machine 100 tracks the locationof the target plant relative to the autonomous machine 100, as describedabove. Thus, the autonomous machine 100 can drive a first weeding module130 to laterally align the reference position of the first weedingmodule 130 with the first opening location, the first weeding module 130arranged in a tool housing 120 behind the light module no in Block S190.More specifically, the autonomous machine 100 can execute closed loopcontrols to match a lateral coordinate of the reference position of theend effector of the tool module 130 to the most recent tracked laterallocation of the target plant or opening location by laterally actuatingthe weeding module 130 within the tool housing 120. In this manner, theautonomous machine 100 can continuously actuate the weeding module 130within the tool module 130 from when the autonomous machine 100initially detects a target plant until a time at which an openinglocation for the target plant is longitudinally aligned with the weedingmodule 130.

When tracking multiple successive target plants within the same croprow, the autonomous machine 100 can laterally align with the targetplant immediately in front of the tool module 130 until the tool module130 has performed an agricultural operation on or around the targetplant. Once the autonomous machine 100 executes the agriculturaloperation, the autonomous machine 100 can actuate the weeding module 130to laterally align with the next plant in the crop row.

Thus, the autonomous machine 100 can: autonomously navigate along afirst crop row comprising the first ground area; and, in response tolateral alignment of the lateral position of the first tool module 130and the real lateral location of the first target plant and longitudinalalignment of a longitudinal position of the first tool module 130 and areal longitudinal location of the first target plant, execute anagricultural operation on the first target plant.

Additionally, when tracking multiple target plants, the autonomousmachine 100 can: detect a second target plant in the first image; accessa lateral pixel location of the second target plant in the first image;autonomous machine 100 based on the lateral pixel location of the secondtarget plant and the first depth profile; and actuate a second toolmodule 130 in the set of tool modules 130 to a lateral positionlaterally aligned with the real lateral location of the second targetplant.

8. Weeding Blade Depth Control

In one implementation, in implementations of the autonomous machineincluding a weeding module, the autonomous machine can utilizedifferences in depth measurements originating from the depth sensor 116and the weeding module to calculate a depth of the blades 132 of theweeding modules. Due to variability in soil quality and other factors,blades 132 of the weeding module can sink into the soil at variousdepths even when the same amount of downward force is applied to theweeding module (e.g., via an active suspension system 134). Theautonomous machine can capture the magnitude of this sinking effect bymeasuring the extension distance of the weeding module and calculating adifference between the extension distance and the depth value recordedby the depth sensor 116 proximal to the lateral location of the toolmodule 130. The autonomous machine 100 can then identify this differenceas the depth of the blades 132 of the weeding module and implementclosed loop controls to maintain a specific blade depth beneath theplant bed (e.g., via an active suspension system 134).

Thus, the autonomous machine 100 can: record an extension distance of afirst weeding module in the set of tool modules 130; and estimate afirst depth of blades 132 of the first weeding module based on adifference between the depth of the subregion of the ground area and theextension distance of the first weeding module.

Additionally, the autonomous machine 100 can: receive and input from auser specifying a set depth for the blades 132 of the first weedingmodule; and actuate a suspension of the first weeding module accordingto feedback control based on the first depth of the blades 132 of thefirst weeding module and the set depth of the blades 132 of the firstweeding module.

In one implementation, the autonomous machine 100 can automaticallyestimate the maturity of an identified target plant according to machinelearning techniques. The autonomous machine 100 can access (e.g., via alookup table) a blade depth appropriate for the plant at the estimatedmaturity. The autonomous machine 100 can then adjust the blade depth ofthe weeding modules in real-time based on the estimated maturity of thetarget plants. Thus, the autonomous machine 100 can: estimate a maturityof the first target plant based on the first image; and modify the setdepth for the blades 132 of the first weeding module based on thematurity of the first target plant.

9. Depth Sensor 116 Error Detection

In one implementation, the autonomous machine 100 can utilize thedifference in depth between a depth value measured at the depth sensor116 and the depth according to a surface profile in order to identifyerroneous depth sensor 116 readings and, instead of adjusting thesurface profile according to the depth value, utilize the surfaceprofile as the depth profile. In this implementation, the autonomousmachine 100 can compare the difference in depth measurements to athreshold difference, which may correspond to a maximum realisticdifference limited, for example, by the suspension of the tool module130, or any other physical or environmental constraint. Upon detectingthat the difference in depth measurements has exceeded this threshold,the autonomous machine 100 can identify a malfunction of the depthsensor 116 and utilize an alternative form of depth measurement (e.g.,by utilizing the extension distances of the tool modules 130exclusively).

Thus, the autonomous machine 100 can, in response to the differencebetween the depth of the subregion of the ground area and the extensiondistance of the first weeding module exceeding a threshold difference:identify malfunction of the depth sensor 116; estimate a second depthprofile of the plant bed surface proximal the first target plant basedon the first extension distance of each tool module 130 in the set oftool modules 130 and the first lateral position of each tool module 130in the set of tool modules 130; and update the real lateral location ofthe first target plant relative to the ground-facing camera 112 based onthe lateral pixel location of the first target plant and the seconddepth profile.

10. Toolbar Adjustment

In one variation, the autonomous machine 100 can adjust the heightand/or tilt of the toolbar based on an estimated surface profile of theplant bed surface generated by the aforementioned method S100.Generally, the autonomous machine 100 adjusts the height and/or tilt ofthe toolbar to maintain contact between the tool modules 130 and theplant bed surface as the autonomous machine 100 draws the tool modules130 along the plant bed. The autonomous machine 100 can record theextension distances relative to a “home” position of the suspensionsystem 134 of each tool module 130. The home position of the suspensionsystem 134 can correspond to the middle of the tool module's range ofvertical motion relative to the toolbar or any other preferable pointwithin the range of motion of the tool module 130. In oneimplementation, the autonomous machine 100 adjusts the height of thetoolbar according to the average extension distance of the set of toolmodules 130 mounted to the toolbar. For example, if the averageextension distance is two inches beyond the home position, then theautonomous machine 100 can lower the toolbar two inches, such that theaverage extension distance of the set of tool modules 130 is closer tothe home position of the set of tool modules 130.

The autonomous machine 100 can also adjust the tilt of the toolbar tomatch the tilt of the plant bed surface below. For example, if, based onthe extension distances measured from the set of tool modules 130, theplant bed is determined to have a tilt of two degrees, then theautonomous machine 100 can adjust the tilt of the toolbar to twodegrees. The autonomous machine 100 can determine the tilt of the plantbed surface by plotting each extension distance versus the lateralposition of the tool module 130 from which the extension distance wasmeasured and performing a linear regression. The slope of the resultinglinear fit can then be set as the tilt for the toolbar (if the value iswithin the toolbar's range of motion).

Therefore, the autonomous machine 100 can: actuate the toolbar from thefirst inclination of the toolbar to a second inclination of the toolbar,the second inclination substantially matching an average inclination ofthe surface profile within a range of motion of the toolbar; and actuatethe toolbar from the first height of the toolbar to a second height ofthe toolbar, the second height minimizing, for the set of tool modules130, a difference between an extension distance of each tool module 130at the second height and a middle of a range of extension of each toolmodule 130.

However, the autonomous machine 100 can actuate the toolbar in any otherway in response to extension distances recorded at the set of toolmodules 130.

11. Other Operations

The autonomous machine 100 can implement similar methods and techniquesto set timers for actuating tool modules 130 of other types, to setlateral positions of these tool modules 130 for passing target plants,and to thus control actuation of these tool modules 130 in order toselectively address target plants and non-target plants (e.g., weeds) inan agricultural field. For example, the autonomous machine 100 canimplement the foregoing methods and techniques to: set a lateral offsetfor a watering module in the tool housing 120 to align a watering headin the watering module to a next target plant in a crop row; set anactivation timer for dispensing water from the watering module directlyonto or in the immediate vicinity of the next target plant; and set adeactivation timer for ceasing dispensation of water from the wateringmodule once the target plant passes the watering module. In a similarexample, the autonomous machine 100 can implement the foregoing methodsand techniques to: set a lateral offset for a fertilizer module in thetool housing 120 to align a spray head in the fertilizer module to anext target plant in a crop row; set an activation timer for dispensingfertilizer from the fertilizer module directly onto or in the immediatevicinity of the next target plant; and set a deactivation timer forceasing dispensation of fertilizer from the fertilizer module once thetarget plant passes the fertilizer module.

The systems and methods described herein can be embodied and/orimplemented at least in part as a machine configured to receive acomputer-readable medium storing computer-readable instructions. Theinstructions can be executed by computer-executable componentsintegrated with the application, applet, host, server, network, website,communication service, communication interface,hardware/firmware/software elements of a user computer or mobile device,wristband, smartphone, or any suitable combination thereof. Othersystems and methods of the embodiment can be embodied and/or implementedat least in part as a machine configured to receive a computer-readablemedium storing computer-readable instructions. The instructions can beexecuted by computer-executable components integrated bycomputer-executable components integrated with apparatuses and networksof the type described above. The computer-readable medium can be storedon any suitable computer readable media such as RAMs, ROMs, flashmemory, EEPROMs, optical devices (CD or DVD), hard drives, floppydrives, or any suitable device. The computer-executable component can bea processor but any suitable dedicated hardware device can(alternatively or additionally) execute the instructions.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the embodiments of the invention without departing fromthe scope of this invention as defined in the following claims.

I claim:
 1. A method comprising, at an autonomous machine: capturing afirst image of a ground area of a plant bed surface via a ground-facingcamera arranged on the autonomous machine; detecting a first targetplant in the first image; accessing a lateral pixel location of thefirst target plant in the first image; via a depth sensor arranged onthe autonomous machine, estimating a depth of a subregion of the groundarea; for each tool module in a set of tool modules in contact with theplant bed surface and arranged behind the ground-facing camera relativeto a direction of forward motion of the autonomous machine: capturing anextension distance of the tool module; and capturing a lateral positionof the tool module relative to the ground-facing camera; estimating asurface profile of the plant bed surface based on the extension distanceof each tool module in the set of tool modules and the lateral positionof each tool module in the set of tool modules; estimating a depthprofile based on the surface profile and the depth of the subregion ofthe ground area; estimating a real lateral location of the first targetplant relative to the autonomous machine based on the lateral pixellocation of the first target plant and the depth profile of the plantbed surface; and driving a first tool module in the set of tool modulesto a lateral position laterally aligned with the real lateral locationof the first target plant.
 2. The method of claim 1, wherein estimatingthe depth profile based on the surface profile and the depth of thesubregion of the ground area comprises: identifying a coincident sectionof the surface profile laterally coincident with the subregion of theground area; and defining the depth profile as a shifted version of thesurface profile comprising the coincident section with an average depthequal to the depth of the subregion of the ground area.
 3. The method ofclaim 1, wherein detecting the depth of the subregion of the ground areacomprises: at the depth sensor, capturing a depth image of the subregionof the ground area; identifying a ground region of the depth image; andcalculating an average depth of the ground region based on the groundregion of the depth image.
 4. The method of claim 3, further comprising:identifying a plant-occupying region of the depth image; and calculatinga plant height based on the plant-occupying region of the depth image.5. The method of claim 4, wherein estimating the real lateral locationof the first target plant relative to the ground-facing cameracomprises: accessing a projection in three-dimensional space relative tothe ground-facing camera corresponding to the lateral pixel location ofthe first target plant; subtracting the plant height from the depthprofile to calculate an approximate depth of the first target plant; andestimating the real lateral location along the projection, the reallateral location corresponding to the approximate depth of the firsttarget plant.
 6. The method of claim 1, further comprising: at thesecond time, capturing an extension distance of a first weeding modulein the set of tool modules; and estimating a first depth of blades ofthe first weeding module based on a difference between the depth of thesubregion of the ground area and the extension distance of the firstweeding module.
 7. The method of claim 6, further comprising: receivinga set depth for the blades of the first weeding module; and actuating asuspension of the first weeding module according to feedback controlbased on the first depth of the blades of the first weeding module andthe set depth of the blades of the first weeding module.
 8. The methodof claim 7, further comprising: estimating a maturity of the firsttarget plant based on the first image; and modifying the set depth forthe blades of the first weeding module based on the maturity of thefirst target plant.
 9. The method of claim 6, further comprising: inresponse to the difference between the depth of the subregion of theground area and the extension distance of the first weeding moduleexceeding a threshold difference: identifying malfunction of the depthsensor; estimating a second depth profile of the plant bed surfaceproximal the first target plant based on the first extension distance ofeach tool module in the set of tool modules and the first lateralposition of each tool module in the set of tool modules; and updatingthe real lateral location of the first target plant relative to theground-facing camera based on the lateral pixel location of the firsttarget plant and the second depth profile.
 10. The method of claim 9,wherein estimating the second depth profile comprises: for each toolmodule in the set of tool modules, defining a contact point of the toolmodule at an intersection of the first extension distance of the toolmodule and the first lateral position of the tool module to generate aset of contact points; and interpolating between the set of contactpoints to generate the second depth profile.
 11. A method comprising, atan autonomous machine: capturing a first image of a ground area of aplant bed surface via a ground-facing camera arranged on the autonomousmachine; detecting a location of a first target plant in the firstimage; via a depth sensor arranged on the autonomous machine, capturinga depth image of the ground area; identifying a ground region of thedepth image based on the first image of the ground area, the groundregion excluding a plant-occupying region of the depth image;calculating a depth of the plant bed surface based on the ground regionof the depth image; via a first tool module in contact with the plantbed surface and arranged behind the ground-facing camera relative to adirection of forward motion of the autonomous machine, capturing a firstextension distance of the first tool module; estimating a first tooldepth below the plant bed surface based on a difference between thedepth of the plant bed surface and the extension distance of the firsttool module; and in response to detecting that the tool depth isdifferent from a set tool depth, actuating a first suspension of thefirst tool module toward the set tool depth.
 12. The method of claim 11:wherein capturing the extension distance of the first tool modulecomprises, for each tool module in a set of tool modules comprising thefirst tool module: capturing an extension distance of the tool module;and capturing a lateral position of the tool module relative to theground-facing camera; and further comprising: estimating a surfaceprofile of the plant bed surface based on the extension distance of eachtool module in the set of tool modules and the lateral position of eachtool module in the set of tool modules; estimating a depth profile basedon the surface profile and the depth of the subregion of the groundarea; estimating a real lateral location of the first target plantrelative to the autonomous machine based on the lateral pixel locationof the first target plant and the depth profile of the plant bedsurface; and driving a first tool module in the set of tool modules to alateral position laterally aligned with the real lateral location of thefirst target plant.
 13. The method of claim 12: further comprising:identifying the plant-occupying region of the depth image based on thelocation of the first target plant and the first image of the groundarea; and calculating a plant height of the first target plant based onthe plant-occupying region of the depth image; and wherein estimatingthe real lateral location of the first target plant relative to theautonomous machine based on the lateral pixel location of the firsttarget plant and the depth profile of the plant bed surface comprisesestimating the real lateral location of the first target plant relativeto the autonomous machine based on the lateral pixel location of thefirst target plant, the depth profile of the plant bed surface, and theplant height of the first target plant.
 14. The method of claim 11:wherein capturing the extension distance of the first tool modulecomprises, for each tool module in a set of tool modules comprising thefirst tool module: capturing an extension distance of the tool module;and capturing a lateral position of the tool module relative to theground-facing camera; further comprising, for each tool module in theset of tool modules, estimating a tool depth below the plant bed surfacebased on the extension distance of the tool module, the depth image, thelateral position of the tool module; and wherein, in response todetecting that the first tool depth is different from the set tooldepth, actuating the first suspension of the first tool module towardthe set tool depth comprises, for each tool module in the set of toolmodules, in response to detecting the tool depth of the tool moduledifferent from the set tool depth, actuating a suspension of the toolmodule toward the set tool depth.
 15. The method of claim 11: wherein,via the first tool module in contact with the plant bed surface andarranged behind the ground-facing camera relative to the direction offorward motion of the autonomous machine, capturing the extensiondistance of the first tool module comprises, via a first weeding modulein contact with the plant bed surface and arranged behind theground-facing camera relative to the direction of forward motion of theautonomous machine, capturing the extension distance of the firstweeding module; wherein estimating the first tool depth below the plantbed surface based on the difference between the depth of the plant bedsurface and the extension distance of the first tool module comprisesestimating a first weeding blade depth below the plant bed surface basedon a difference between the depth of the plant bed surface and theextension distance of the first weeding module; and wherein, in responseto detecting that the tool depth is different from the set tool depth,actuating the first suspension of the first tool module toward the settool depth comprises, in response to detecting that the weeding bladedepth is different from a set weeding blade depth, actuating a firstsuspension of the first weeding module toward the set weeding bladedepth.
 16. The method of claim 15, further comprising: estimating amaturity of the first target plant based on the first image; and settingthe set weeding blade depth based on the maturity of the first targetplant.
 17. A method comprising, at an autonomous machine: capturing afirst image of a ground area of a plant bed surface via a ground-facingcamera arranged on the autonomous machine; detecting a location of afirst target plant in the first image; via a depth sensor arranged onthe autonomous machine, capturing a depth image of the ground area;identifying a ground region of the depth image based on the first imageof the ground area, the ground region excluding a plant-occupying regionof the depth image; calculating a depth profile of the plant bed surfacebased on the ground region of the depth image; for each tool module in aset of tool modules in contact with the plant bed surface and arrangedbehind the ground-facing camera relative to a direction of forwardmotion of the autonomous machine: capturing an extension distance of thetool module; and capturing a lateral position of the tool modulerelative to the ground-facing camera; estimating a tool depth below theplant bed surface based on the extension distance of the tool module,the lateral position of the tool module, and the depth image; and inresponse to detecting that the tool depth is different from a set tooldepth, actuating a suspension of the tool module toward the set tooldepth.
 18. The method of claim 17, wherein estimating the tool depthbelow the plant bed surface based on the extension distance of the toolmodule, the lateral position of the tool module, and the depth imagecomprises: extracting a depth value from the depth image at the lateralposition of the tool module; and calculating a difference between thedepth value and the extension distance of the tool module.
 19. Themethod of claim 17: wherein, for each tool module in the set of toolmodules, capturing the extension distance of the tool module comprises,for each weeding module in a set of weeding modules, capturing anextension distance of the weeding module; wherein, for each tool modulein the set of tool modules, capturing the lateral position of the toolmodule relative to the ground-facing camera comprises, for each weedingmodule in the set of weeding modules, capturing a lateral position ofthe weeding module relative to the ground-facing camera; wherein, foreach tool module in the set of tool modules, estimating the tool depthbelow the plant bed surface based on the extension distance of the toolmodule, the lateral position of the tool module, and the depth imagecomprises, for each weeding module in the set of weeding modules,estimating a weeding blade depth below the plant bed surface based onthe extension distance of the weeding module, the lateral position ofthe weeding module, and the depth image; and wherein, for each toolmodule in the set of tool modules, in response to detecting that thetool depth is different from the set tool depth, actuating thesuspension of the tool module toward the set tool depth comprises, foreach weeding module in the set of weeding modules, in response todetecting the weeding blade depth different from a set weeding bladedepth, actuating a suspension of the weeding module toward the setweeding blade depth.
 20. The method of claim 19, further comprising:estimating a maturity of the first target plant based on the firstimage; and setting the set weeding blade depth based on the maturity ofthe first target plant.